You may not be able to walk between the beach and the caves - too far and too steep. The walk between Caves House and the Caves is a nice bush walk. A car or a tour are the best options to see the area.
Yallingup